1. Field of the Invention
The present invention relates to a three-dimensional (3D) graphic processing technology of using camera preview images, and more particularly, to a three-dimensional (3D) graphic processing system and method capable of utilizing camera preview images in which the camera preview images are stored in a texture memory and then the stored camera preview images are used as a texture in a 3D graphic processor.
2. Description of the Related Art
Recently, digital cameras are very popular as much as a newly coined word called a “DICA tribe” has appeared in order to call people which enjoy the digital cameras. Particularly, according to development of image sensors with high sensitivity and ultra-small digital camera modules, ultra-small digital cameras can be mounted on mobile communications terminals or personal digital assistants (PDAs). Furthermore, according to appearance of high-performance ultra-small digital camera modules which are not behind general digital cameras, the digital cameras put down roots as essential items of mobile communications terminals together with color liquid crystal displays (LCDs).
The mobile communications terminals mounting the ultra-small digital cameras (called camera phones) conveniently take a picture of an object. Then, the camera phones use the taken picture as a background picture, or immediately transmit the taken picture to a desired person. Accordingly, the camera phones are very popular. In particular, since the new generation thinking especially the mobile communications terminals as tools of self-expression can express their own personalities, the camera phones become further popular.
In the meantime, digital cameras or digital camcorders as well as the above-described camera phones are provided with a preview function of allowing a photographer to confirm or preview a picture to be photographed through a liquid crystal display (LCD) in advance, respectively.
In order to perform a preview operation, an image sensor scans an image in a Video Graphics Array (VGA) form (that is, 640×480, 320×240). Then, the scanned image is converted into a digital signal. Then, the digital signal is converted into a color signal of a predetermined mode such as YUV or YCbCr and delivered to a main processor in a serial or parallel mode. Thereafter, the main processor converts video data of the YUV or YCbCr color into the color of the other predetermined mode such as RGB (Red, Green, Blue) code and stores the converted result in a storage device. The main processor resizes the converted result according to the resolution (that is, the size) of a liquid display device (LCD) and outputs the resized result as a preview.
The conventional preview mode has the problem that the size of video data which the image sensor scans is greater than the resolution (the size) of the general display device such as a liquid crystal display (LCD), to thus let the video data to previewed after passing through conversion of the resolution (the size) (that is, resizing), and to thereby cause the load of the main processor to be enlarged and the preview speed to be lowered.
Particularly, when a large capacity of image data is processed because the main processor performs various tasks in the camera phone, a processing speed falls down very much. The space of a storage device for storing video data of which the capacity is large is wasted.
In Korea Patent Laid-open Publication No. 2004-106658 was proposed an image data high speed preview method in consideration of the above-described problem. Here, a processor delivers the resolution of a display device such as a LCD to an image sensor, so that the image sensor scans an image according to the resolution of the display device, and then delivers the scanned image to the processor so that the processor outputs on the display device without any conversion of the resolution to thus enable a user to enjoy a high speed preview of image data.
Moreover, in Korea Patent Laid-open Publication No. 2005-5102 was disclosed a video data preview automatic control method. Here, when video data is previewed in digital cameras, and digital camcorders or mobile communications terminals having a digital photographing function, a preview is automatically stopped to reduce unnecessary power consumption if it is determined that a preview state was left alone over a predetermined time.
However, the conventional art displays the images obtained through the image sensor on the display device as they are, but does not present a measure which utilizes the images in various forms in a graphic processor.
In the meantime, FIG. 1 shows the whole structure of a conventional camera chip having no graphic processor. The conventional camera chip 1 includes a host interface/control register (Host I/F & Control REG) 5, a sensor interface/ISP (Image Signal Processor) 6, a preview processor 7, a JPEG (Joint Photographic coding Experts Group) CODEC (COmpress/DECompress) 8, a stacked memory 32 and a LCD interface 9.
The host interface/control register (Host I/F & Control REG) 5, is provided to perform an interface with a host 3 and to store a control signal of the host with respect to the camera chip.
The sensor interface/ISP (Image Signal Processor) 6 receives an input image input from a charge coupled device (CCD) or a CMOS type image sensor 2 for example, to the camera chip 1 and is connected to the host 3 through the host interface/control register 5.
The preview processor 7 performs a preview operation of sending the image received through the sensor I/F & ISP 6 from the sensor 2 immediately to a liquid crystal display (LCD) 4.
The JPEG CODEC 8 compresses the image received from the image sensor by JPEG and decodes JPEG image received from the host 3 to then be sent to the LCD 4.
The stacked memory 32 stores the JPEG compressed image received from the image sensor 2 through a memory interface 31 and JPEG image received from the host 3.
The LCD interface 9 performs an interface between the preview processor 7 and the LCD 4.
When this conventional camera chip provides a preview function of allowing a user to preview and confirm a picture of an object to be photographed through the LCD 4 in advance, conversion of the resolution (the size), that is, resizing is performed, to fit for the display size of the LCD 4 in the preview processor 7, and an input image received from the image sensor 2 is sent immediately to the LCD 4 to then be simply displayed. However, a further expanded function is not supported.
FIG. 2 is a schematic block diagram showing a merged chip including a conventional camera processor and a graphic processor. The merged chip 10 includes a graphic processor 19b for acceleration of 3D graphics in addition to a camera processor 19a which is the same as the camera chip of FIG. 1. The camera processor 19a and the graphic processor 19b perform respectively different operations and output the processed results through a multiplexer 18 to a LCD 4.
The graphic processor 19b includes a host interface (Host I/F) & control register 11, a geometric engine (GE) 12, a rendering engine (RE) 13, a texture memory 14, a memory interface 15, a frame memory 16 and a LCD interface 17, in order to perform the processing for 3D graphics acceleration.
The geometric engine 12 chiefly performs position transformation and lighting calculation of a polygon necessary for 3D graphic processing so that a point (vertex) on a basis coordinate system is converted into a coordinate positioned through an actual eye.
The rendering engine 13 actually draws pictures in units of a pixel based on the coordinate generated from the geometric engine 12.
The texture memory 14 memorizes texture data.
In the configuration of FIG. 2, the host interface & control registers 5 and 11 and the LCD interfaces 9 and 17 can be simplified into a single block, respectively, so that the conventional camera chip 19a and the graphic processor 19b share the simplified blocks. Otherwise, as described above, the host interface & control registers 5 and 11 and the LCD interfaces 9 and 17 can be separately designed.
However, even in the case of the merged chip having the camera processor 19a and the graphic processor 19b, the other functions are not supported except that the preview image of the camera is directly displayed on the LCD 4 and thus the camera input image is only displayed on the LCD 4 of the rectangular shape. Moreover, in the case of the conventional graphic process of writing texture data in the texture memory 14 through only a texture write pass, only the graphic using the texture which is considered during the manufacture in the contents can be supported.
FIG. 3 is a block diagram showing a modified structure of FIG. 2 that uses a single LCD interface I/F. In FIG. 3, the output of a preview processor 7 and the output of a graphic processor 19b are selected in a multiplexer 18a and the selected result is output to a LCD 4 through a LCD interface 17a. Here, since the FIG. 3 structure is the same as the FIG. 2 structure, except for the above-described difference of the position of the LCD 4 between FIGS. 2 and 3, the same reference numerals are assigned to the same elements in FIGS. 2 and 3. That is, according to a graphic enable signal from a host, the multiplexer 18a outputs a graphic signal, or a preview image of the camera to the LCD interface 17a. Therefore, the modified signal processing circuit of FIG. 3 performs the same operations as those of FIG. 2, the detailed description thereof will be omitted.
As described above, the camera processor and the graphic processor are not connected each other but separately operate. A support of an extended camera function is essential at a situation where only the camera function is no more competitive power. Accordingly, a graphic processing system of a new concept capable of using a realistic camera input image as a texture at the side of the graphic function is required.